Dynamic Data Relevance Estimation by Exploring Models (D2REEM)

Analysts in many areas of national security face a massive (high volume), dynamically changing (high velocity) flood of possibly relevant information. Identifying reasonable suspects confronts a tension between data that is too atomic to be diagnostic and knowledge that is too complex to guide search. DREEM (Dynamic Data Relevance Estimation by Exploring Models) is a knowledge-based metaheuristic that uses stochastic search of a graph-based semantic model to guide successive queries of high-volume, high-velocity data. We motivate DREEM by considering the nature of knowledge-based search in highvolume, high-velocity data and reviewing current tools. We then outline the DREEM metaheuristic and describe the state of progress in applying it to a range of model types, including geospatial movement, behavioral models, discourse models, narrative generators, and social networks. Finally, we outline work that needs to be done to advance the DREEM agenda. Keywords—retrieval, querying, semantic models, big data, stochastic search, any-time methods